Proximal boosting and its acceleration

Gradient boosting is a prediction method that iteratively combines weak learners to produce a complex and accurate model. From an optimization point of view, the learning procedure of gradient boosting mimics a gradient descent on a functional variable. This paper proposes to build upon the proximal point algorithm when the empirical risk to minimize is not differentiable to introduce a novel boosting approach, called proximal boosting. Besides being motivated by non-differentiable optimization, the proposed algorithm benefits from Nesterov's acceleration in the same way as gradient boosting [Biau et al., 2018]. This leads to a variant, called accelerated proximal boosting. Advantages of leveraging proximal methods for boosting are illustrated by numerical experiments on simulated and real-world data. In particular, we exhibit a favorable comparison over gradient boosting regarding convergence rate and prediction accuracy

Inference of multivariate exponential Hawkes processes with inhibition and application to neuronal activity

The multivariate Hawkes process is a past-dependent point process used to model the relationship of event occurrences between different phenomena.Although the Hawkes process was originally introduced to describe excitation effects, which means that one event increases the chances of another occurring, there has been a growing interest in modelling the opposite effect, known as inhibition.In this paper, we focus on how to infer the parameters of a multidimensional exponential Hawkes process with both excitation and inhibition effects. Our first result is to prove the identifiability of this model under a few sufficient assumptions. Then we propose a maximum likelihood approach to estimate the interaction functions, which is, to the best of our knowledge, the first exact inference procedure in the frequentist framework.Our method includes a variable selection step in order to recover the support of interactions and therefore to infer the connectivity graph.A benefit of our method is to provide an explicit computation of the log-likelihood, which enables in addition to perform a goodness-of-fit test for assessing the quality of estimations.We compare our method to standard approaches, which were developed in the linear framework and are not specifically designed for handling inhibiting effects.We show that the proposed estimator performs better on synthetic data than alternative approaches. We also illustrate the application of our procedure to a neuronal activity dataset, which highlights the presence of both exciting and inhibiting effects between neurons.Comment: Statistics and Computing, 202

Comparaison de descripteurs pour la classification de décompositions parcimonieuses invariantes par translation

National audienceNous étudions les descripteurs adaptés à la classification de décompositions parcimonieuses invariantes par translation. Nous comparons les différents descripteurs de l'état de l'art sur les mêmes données et avec le même classifieur, ce qui permet d'évaluer leurs efficacités et nous testons aussi leur robustesse à la translation. Grâce à un nouveau fenêtrage, une famille de nouveaux descripteurs est proposée, dépassant l'état de l'art tout en étant robuste à la translation

Outils d'apprentissage automatique pour la reconnaissance de signaux temporels

The work presented here tackles two different subjects in the wide thematic of how to build a numerical system to recognize temporal signals, mainly from limited observations. The first one is automatic feature extraction. For this purpose, we present a column generation algorithm, which is able to jointly learn a discriminative Time-Frequency (TF) transform, cast as a filter bank, with a support vector machine. This algorithm extends the state of the art on multiple kernel learning by non-linearly combining an infinite amount of kernels. The second direction of research is the way to handle the temporal nature of the signals. While our first contribution pointed out the importance of correctly choosing the time resolution to get a discriminative TF representation, the role of the time is clearly enlightened in early recognition of signals. Our second contribution lies in this field and introduces a methodological framework for early detection of a special event in a time-series, that is detecting an event before it ends. This framework builds upon multiple instance learning and similarity spaces by fitting them to the particular case of temporal sequences. Furthermore, our early detector comes with an efficient learning algorithm and theoretical guarantees on its generalization ability. Our two contributions have been empirically evaluated with brain-computer interface signals, soundscapes and human actions movies.Les travaux présentés ici couvrent deux thématiques de la reconnaissance de signaux temporels par des systèmes numériques dont certains paramètres sont inférés à partir d’un ensemble limité d’observations. La première est celle de la détermination automatique de caractéristiques discriminantes. Pour ce faire, nous proposons un algorithme de génération de colonnes capable d’apprendre une transformée Temps-Fréquence (TF), mise sous la forme d’un banc de filtres, de concert à une machine à vecteurs supports. Cet algorithme est une extension des techniques existantes d’apprentissage de noyaux multiples, combinant de manière non-linéaire une infinité de noyaux. La seconde thématique dans laquelle s’inscrivent nos travaux est l’appréhension de la temporalité des signaux. Si cette notion a été abordée au cours de notre première contribution, qui a pointé la nécessité de déterminer au mieux la résolution temporelle d’une représentation TF, elle prend tout son sens dans une prise de décision au plus tôt. Dans ce contexte, notre seconde contribution fournit un cadre méthodologique permettant de détecter précocement un événement particulier au sein d’une séquence, c’est à dire avant que ledit événement ne se termine. Celui-ci est construit comme une extension de l’apprentissage d’instances multiples et des espaces de similarité aux séries temporelles. De plus, nous accompagnons cet outil d’un algorithme d’apprentissage efficace et de garanties théoriques de généralisation. L’ensemble de nos travaux a été évalué sur des signaux issus d’interfaces cerveau-machine, des paysages sonores et des vidéos représentant des actions humaines

Early and reliable event detection using proximity space representation

Conference of 33rd International Conference on Machine Learning, ICML 2016 ; Conference Date: 19 June 2016 Through 24 June 2016; Conference Code:124527International audienceLet us consider a specific action or situation (called event) that takes place within a time series. The objective in early detection is to build a decision function that is able to go off as soon as possible from the onset of an occurrence of this event. This implies making a decision with an incomplete information. This paper proposes a novel framework that i) guarantees that a de-tection made with a partial observation will also occur at full observation of the time-series; ii) incorporates in a consistent manner the lack of knowledge about the minimal amount of information needed to make a decision. The proposed detector is based on mapping the temporal sequences to a landmarking space thanks to appropriately designed similarity functions. As a by-product, the framework benefits from a scalable training algorithm and a theoretical guarantee concerning its generalization ability. We also discuss an important improvement of our framework in which decision function can still be made reliable while being more expressive. Our experimental studies provide compelling results on toy data, presenting the trade-off that occurs when aiming at accuracy, earliness and reliability. Results on real physiological and video datasets show that our proposed approach is as accurate and early as state-of-the-art algorithm, while ensuring reliability and being far more efficient to learn

Early frame-based detection of acoustic scenes

International audienceLet us consider a specific acoustic scene appearing in a continuous audio stream recorded while making a trip a in city. In this work, we aim at detecting at the earliest opportunity the several occurrences of this scene. The objective in early detection is then to build a decision function that is able to go off as soon as possible from the onset of a scene occurrence. This implies making a decision with an incomplete information. This paper proposes a novel framework in this area that i) can guarantee the decision made with a partial observation to be the same as the one with the full observation; ii) incorporates in a non-confusing manner the lack of knowledge about the minimal amount of information needed to make a decision. The proposed detector is based on mapping the temporal sequences to a landmarking space thanks to appropriately designed similarity functions. As a by-product, the built framework benefits from a scalable learning problem. A preliminary experimental study provides compelling results on a soundscape dataset